Papers with up decoding
Infusing Sequential Information into Conditional Masked Translation Model with Self-Review Mechanism (2020.coling-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive models generate target words in parallel, but with a large latency due to the left-to-right dependency. |
| Approach: | They propose to train a conditional masked translation model and refine results within several iterations to remedy a flawed translation by non-autoregressive models. |
| Outcome: | The proposed model outperforms state-of-the-art models by over 1 BLEU while using less training computations. |
EDITOR: An Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints (2021.tacl-1)
Copied to clipboard
| Challenge: | Empirically, EDITOR uses soft lexical constraints more effectively than the Levenshtein Transformer while speeding up decoding dramatically compared to constrained beam search. |
| Approach: | They propose an Edit-Based TransfOrmer with Repositioning that integrates lexical preferences into output sequences by iterative editing hypotheses. |
| Outcome: | The proposed model uses soft lexical constraints more effectively than the Levenshtein Transformer while speeding up decoding dramatically compared to constrained beam search. |
Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement (D18-1)
Copied to clipboard
| Challenge: | Despite its success, neural autoregressive modeling has its weakness in decoding, i.e., finding the most likely sequence. |
| Approach: | They propose a conditional non-autoregressive neural sequence model based on iterative refinement based upon latent variable models and conditional denoising autoencoders. |
| Outcome: | The proposed model significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart. |